With the Augmented Reality technology, a video feed, either live or pre-recorded, or a series of related images may be augmented with one or more virtual objects. The video feed, the frames of the video feed, or the related images that are augmented with virtual objects may be hereinafter referred to as image targets. The virtual objects are superposed on the image targets so that the virtual objects appear as if they are part of the real-world scenes in the image targets. The pose of the image targets may be tracked with known methods, and the pose of the virtual objects may be adjusted accordingly to maintain the realism of the virtual objects.
When lighting conditions change in the image target, a known technique named Color Transfer may be used to maintain the realism of the virtual object by adjusting the colors of the virtual object so that the colors of the virtual object reflect the lighting condition change in the image target. An image target frame with a neutral lighting condition may be used as the ground truth, i.e., the baseline, for measuring the change in the lighting condition. Broadly speaking, the Color Transfer technique measures the color differences between the colors of the incoming image target frame and the colors of the ground truth image target frame, and adjusts the colors of the virtual objects accordingly. A Color Transfer function may be derived and utilized in the process, and the Color Transfer function may take into account colors of the ground truth image target frame, colors of the incoming image target frame, and colors of the virtual objects before adjustment, and provide suitable adjusted colors for the virtual objects.
Known Color Transfer technique may result in incorrect, unexpected, or less-than-ideal results under certain circumstances. For example, it may be susceptible to occlusion: the color difference measurements in the Color Transfer process may be incorrect or conflicting when, for example, an occluding object such as a hand not present in the ground truth image target frame appears in the incoming image target frame. The colors of the occluding object may be compared to the colors of the ground truth by the known Color Transfer process to derive color differences when colors of the occluding object and colors of the ground truth are not comparable as they are not associated with the same physical object, resulting in incorrect color difference measurements. Using these incorrect color difference measurements, the known Color Transfer technique may apply a wrong color adjustment to the virtual object. The known Color Transfer technique may also generate less-than-ideal results when there are scale changes in the incoming image target frames (e.g., when the incoming image target frames represent a zoomed in or zoomed out view).